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The manner in which groups of neurons represent events in the external world is fundamental to 
neuroscience. Here, we analyze the population code of the retina during naturalistic stimulation 
and show that the information conveyed by specific multi-neuronal firing patterns can be very 
different from the sum of the patterns parts. Synchronous spikes convey more information than 
either of the participating cells, but almost always less than their sum, making them redundant 
coding symbols. Surprisingly, patterns of spiking and silence are mostly synergistic carrying 
information that is accessible only by observing the whole pattern of activity, rather than its 
components, and signifying unique features in the stimulus. These results demonstrate that the 
retina uses a combinatorial code and that the brain can benefit significantly from recognizing 
multi-neuronal firing patterns. 



While much of our understanding of the brain is de- 
rived from studies of single neurons, it is clear that neu- 
ral systems generally rely on populations of neurons to 
represent stimuli and to direct motor outputs. In partic- 
ular, concerted firing among groups of neurons has been 
observed in many different neural systems, often occur- 
ring more frequently than would be expected if neurons 
were acting independently However, many dif- 

ferent views of how concerted firing patterns represent 
information collectively have beenproposed, even within 
the same brain region In 
one view, the unreliability of single neurons requires that 
many neurons encode the same information, and only 
by averaging over large groups can encoded information 
be extracted reliably Another hypothesis is that 

the information conveyed by multi-neuronal spiking pat- 
terns is simply the sum of information that each of the 
cells conveys on its own; this view is related to ideas of 
redundancy reduction il2il, dccorrclation (13) and inde- 
pendent component analysis QA:, J 5), which have been 
suggested as design principles for efficient codi ng. Fi- 
nally, there is the possibility of concerted coding |l9), in 
which synchronous multi-neuronal activity patterns can 
be synergistic, conveying more information than the sum 
of contributions from the individual neurons. Such a cod- 
ing scheme has the capacity to multiplex many distinct 
messages onto the axons of a small number of neurons. 

To assess these different coding schemes, the average 
amount of information that small groups of cells convey 
about natural and artificial stimuli has been compared 
to the sum of single cell contributions. This analysis has 
been carried out in ma ny sy stems vertebrate retina ifiol) . 
primary visual cortex l)l7l : IT^ . primary somatosensory 
cortex |19j) , the auditory pathway l)2lh and motor cortex 
123; 112) ■ Typically, the deviation from independence has 
been small, and redundancy is most common. However, 
synergistic response patterns may simply be rare events 
that have only a small effect on the average mutual in- 
formation. In particular, synchronized spiking has been 



shown to be correlated with specific stimulus features or 
motor outputs if^ IS l23t 12^ : l25l). and has therefore been 
suggested as a special symbol in the population code of 
many neural circuits. 

These issues can be addressed by focusing on the infor- 
mation conveyed by specific multi-cell activity patterns 
(f27i) rather than estimating the mutual information be- 
tween stimuli and responses, which is an average over 
all multi-cell patterns This is a natural framework 
in which to analyze a population code, because specific 
firing patterns are the actual neural symbols that down- 
stream circuits encounter in real time. Using this analy- 
sis, we find significant synergy and redundancy in dif- 
ferent patterns of ganglion cell firing. Averaged over 
patterns, these contributions often partially cancel, ex- 
plaining the relatively low values commonly observed in 
previous studies. We study the stimulus features that 
correspond to multi-neuronal firing patterns and relate 
synergy to the functional properties of individual neu- 
rons. Finally, we show that similar properties emerge 
from a generic model of neural responses, suggesting that 
combinatorial coding may exist in other neural circuits, 
too. 



Results 

Simultaneous recordings were made from many gan- 
glion cells in the salamander and guinea pig retinas us- 
ing a multi-electrode array, while artificial and natural 
movies were presented on a computer monitor (see Meth- 
ods). Movie clips 20-30 s long were repeated many times 
(100-150) in order to sample both the diversity and vari- 
ability of ganglion cell firing patterns. An example of the 
spike trains recorded from two cells is shown in Fig. lA. 



Individual Neural Symbols 

The amount of information that the arrival time of a 
neural symbol such as a spike or any other specific pat- 
tern of neuronal activity in an ensemble (see below) con- 
veys about a complex, time-varying stimulus S is given 
by M 



1 



dt 



(1) 



where a is the neural symbol, rc{t) is the time dependent 
rate of the symbols occurrence, and fa- is the time average 
of over the entire duration of the stimulus T . The sym- 
bol rate ra{t) measures the probability per unit time of 
the symbol occurring, and is estimated in the same way 
as we estimate time-dependent spike rates, by forming 
a peri-stimulus time histogram (PSTH) using repeated 
trials of the stimulus. As with the conventional time- 
dependent firing rate for single spikes, modulations in 
this symbol rate reflect reproducible locking of the events 
(7 to dynamic features in the stimulus. Neuronal symbols 
that occur with a rate that is strongly modulated by the 
stimulus convey more information per symbol than those 
with a weakly modulated rate. Notice that equation 1 
depends only on the variation in the time-varying sym- 
bol rate, r^{t), compared with the average symbol rate, 
f^. Using this relationship, we estimated the informa- 
tion carried by different ganglion cell firing patterns (see 
Methods). 

The simplest neural symbol is the single spike, denoted 
as 1. Over the whole set of natural movies, the average 
information conveyed by a single spike in the salaman- 
der was {I{Ia', S))a = 3.6 ± 1.0 bits (mean ± standard 
deviation across the population; n = 204 cells) and in 
the guinea pig was 2.2 ± 1.2 bits (n=59 cells). No clear 
dependence on the particular movie clip was observed. 

For synchronous spiking among pairs of neurons, we 
defined the compound symbol 1a 1b denoting all the 
spikes of cell A that had a spike of cell B within ± 
10 ms (see Fig. lA). This narrow temporal window 
for synchrony matches the time scale over which excess 
synchronous pairs have been observed Q) and also cor- 
responds with the typical duration of excitatory post- 
synaptic currents evoked by AMPA receptors L2&i) . 

The information that synchronous spiking conveyed 
about natural movies was significantly higher than for 
single spikes, {I{Ia "X) 1b; S))^a,b} — 5-3 ± 1-0 bits (Fig 
IC; n=2887 pairs). In particular, for all cell pairs the 
symbol 1a ® 1b was at least as informative, and usually 
much more informative than either 1a or 1 b alone (data 
not shown). Changing the temporal window used to de- 
fine synchrony between ±5 ms and ±50 ms had a neg- 
ligible effect on the symbols information. Interestingly, 
the 1 (X) 1 symbols obeyed a similar relation to that of the 
single spikes: given our time resolution of 20 ms, nearly 
all symbols conveyed an information between 40% and 
70% of the maximum possible information determined 
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FIG. 1 Information carried by single spikes and synchronous 
spike pairs. A. Typical response segment of ganglion cells 
to repeated presentations of a natural movie. Top and bot- 
tom panels show the spikes of two single cells (blue); middle 
panel shows the events of synchronized spiking of these cells 
within ±10 ms (red). B. Post stimulus time histograms of the 
spike rasters in panel A. C, D. Information conveyed by single 
spikes (blue; every dot stands for one ganglion cell) and syn- 
chronized spike pairs (red; every dot stand for one cell pair) 
as a function of symbol rate for the (salamander, guinea pig) . 
Data points shown from 2 different natural movies. Solid 
line marks the upper bound for information transmission by 
a symbol at the given rate with a 20 ms temporal precision 
[28]; dashed lines are 40% and 70% of this maximum for the 
salamander and 25%/70% for the guinea pig. 



by that symbols average event rate (Fig. IC) (29 ) . Sim- 
ilar behavior was found for the guinea pig (Fig. ID), 
but with somewhat higher firing rates (4.2 ± 1.2 Hz vs. 
1.2 ± 0.8 Hz) and lower coding efficiency (33% ± 5% vs. 
55% ±11%). 



Combinatorial Codes 

To analyze the existence and nature of combinatorial 
codes in the retina, we compared the information trans- 
mitted by a compound symbol a multineuronal firing 
pattern to the sum of information values conveyed by 
the components of the compound symbol. In the case of 
two neurons, we define: 

AI{aA ® ctb; 5) = I {a A ® ctb; 5) - I {a a; S) - I(<7b] S) 

(2) 

If A/ is zero, then the components of the compound 
symbol contribute independent (additive) information 
about the visual stimulus. Otherwise, a combinatorial 
code exists, and downstream circuits can benefit by inter- 
preting multi-neuronal firing patterns as distinct coding 
symbols. In particular, synergy (A/ > 0) implies that 
the compound symbol encodes information that is not 



available at the level of single neuron events. 



Synchronous Spiking 

For natural movies, we found that synchronous spikes 
were significantly less informative than expected from 
summing the information contributed by each individ- 
ual spike. The redundancy of synchronous spikes ranged 
up to A/ = —4.1 bits in salamander (-47% of the con- 
stituent symbol information) and -2.7 bits in guinea pig 
(-22%). For cells spaced less than 200 apart, which 
have overlapping receptive field centers, AI = — 1.1±0.84 
bits (n=2887 pairs) for salamander and -0.41 ± 0.72 bits 
(n=766 pairs) for guinea pig. No clear differences were 
found between movie clips (n=9 clips). Synchronized fir- 
ing tended to produce more negative AI values between 
nearby cells than faraway cells (Fig 2 A) . We found a small 
fraction of cell pairs with synergy for synchronous spike 
pairs; in almost all cases, one cell was ON-type and the 
other was OFF-type (data not shown). 

To further interpret these results, we carried out a sim- 
ilar analysis for artificial stimuli. In spatially uniform 
flicker, where all ganglion cells see the same input, the 
largest values of redundancy were similar to those found 
with naturalistic stimulation, but many more cell pairs 
had high values of redundancy (Fig. 2B). This makes 
sense because the strong spatial correlations in natural 
images will cause ganglion cells with highly overlapping 
receptive fields to experience roughly, but not exactly, 
the same visual input. In checkerboard flicker, where 
spatial correlations only extend up to the size of square 
regions (55 /im), synchronous spike pairs were less redun- 
dant, AI = —0.94 ±0.89 bits for cells spaced by less than 
200 fim (Fig. 2B, n=567 pairs). Together these results 
suggest that the degree of redundancy for synchronous 
spike pairs is largely determined by the properties of the 
stimulus. 

The simple correlation structure of artificial movies 
also allows us to relate neural events to the visual stim- 
ulus fSO"). Using spatially uniform flicker, we typically 
found that the symbol-triggered average stimulus (STA) 
for 1 ® 1 symbols was intermediate between the STAs of 
its component spikes. However, the stimuli preceding a 
synchronous spike pair clustered more tightly around the 
STA than for single spikes. Figure 2D shows that the 
variance along the direction in stimulus space defined by 
the STA was consistently smaller for 1 (g) 1 symbols than 
for single spikes (see Methods). These results indicate 
that the extra information conveyed by 1 i8) 1 symbols 
compared to single spikes is due, in part, to increased 
certainty that the stimulus is similar to the STA. 

The fact that most of the 1 ® 1 symbols were less infor- 
mative than their sum needs to be put in context: how 
much information can a synchronous spike pair possibly 
convey? Intuitively, if two neurons each are very pre- 
cisely locked to the stimulus, then there is little scope for 
synchronous spikes to say anything different from that 
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FIG. 2 Synergy of synchronous spike pairs. A. Synergy of 
synchronous spike pairs, 1(8)1, as a function of the distance be- 
tween the receptive field center of these cells (every dot stands 
for one cell pair). Above: for salamander, natural movie 
data is collected over 3 different movies (green); checkerboard 
movie data is collected over 2 different movies (red). Below: 
for guinea pig, natural movie data is collected over 1 movie 
(purple). B. Distribution of synergies for 1(8)1 symbols in sala- 
mander for natural movies (green, 2556 cell pairs), checker- 
board movies (red, 382 pairs) and uniform flicker (blue, 749 
pairs); synergies for 1 (g) 1 symbols in guinea pig for natural 
movies (purple, 766 pairs). C. Actual synergy for 1 (8 1 sym- 
bols plotted versus the upper bound on their synergy derived 
from the PSTH of each cell (see Methods) for natural movies 
(green) and uniform flicker (blue) . Error bars omitted for clar- 
ity. D. Distribution of stimulus certainty for 1 (g) 1 symbols 
divided by the certainty for single spikes. Certainty is deflned 
as the variance of the symbol-triggered stimulus distribution 
(see text). 



said by the single neurons. We explored the potential for 
synergy between two neurons by using an iterative algo- 
rithm for finding the most informative synchronous spike 
train given the firing rate of each individual neuron (see 
Methods). Fig. 2C compares the potential synergy for 
1(8> symbols with their actual value. Under naturalistic 
stimulation, cell pairs had considerable room for synergy 
but instead were mostly redundant (green). Under spa- 
tially uniform flicker, we could not always find a possible 
combination of spikes into synchronous pairs that was 
synergistic (blue) , but all synchronous spike pairs we sig- 
nificantly more redundant than they could have been. 



Spiking and Silence 

In a combinatorial code, the combination of spiking 
in some neurons and silence (non-spiking) in other neu- 
rons can also be an important event. Silence alone is not 
highly informative: the information content of the sym- 
bol is 0.011 ± 0.015 bits for 20 ms of silence and 0.074 
± 0.079 bits for 100 ms of silence (natural movies, sala- 
mander, n=204 cells). However, the symbol 1a^Q + B , 
where cell A fires a spike and cell B does not spike within 
50 ms of cell As spike, often was found to be synergistic 
(A/ > 0) both for natural and for artificial movies (Fig. 
3A) . Importantly, for every ganglion cell, there is another 
cell whose silence adds synergistically to the spiking cell 
(Fig. 3A). Synergy from silence tended to increase when 
the temporal window used to define silence increased; 
we chose 50 ms because this corresponds to the typical 
duration oflSPSs (2^;'3T), consistent with a likely mech- 
anism for detecting combinations of spiking and silence 
(see below, Fig. 6). 

Synergy from silence becomes even more apparent for 
firing patterns with one spiking cell and several silent 
cells: for natural movies, 10 symbols were more 
synergistic than 1 ® , and 1 1?) symbols were 
even more so, with synergy exceeding 2 bits in some cases 
(Fig 3B). Here, synergy from silence is a generalization 
of Eqn. 2: for instance, A/(l (g) 0; 5) = /(I ® 
0; S)-I{1; S") -7(0(8)0; S). The synergy for symbols was 
0.50 ± 0.46 bits (n=6438 quadruplets) for salamander 
and 0.23 ± 0.33 bits (n=2455 quadruplets) for guinea 
pig. The synergistic contribution from the silence of 3 
neurons is roughly equal to the sum of the synergies from 
the silence of each neuron combined pairwise with the 
spiking neuron (Figs. 3C,D). 

To understand why the combination of spiking and si- 
lence can be synergistic, it is instructive to look at exam- 
ples of such coding symbols under random flicker stimu- 
lation. Synergy results when the requirement of silence 
from the second neuron (cell B) effectively vetoes a sub- 
set of the firing events produced by the spiking cell (cell 
A) , while leaving other events essentially unchanged (Fig. 
4A). Because this vetoing operation is stimulus specific, 
the STA of the 100 symbol is qualitatively different from 
the STA of cell A alone (Fig. 4B). In fact, the visual mes- 
sage represented by a 1 symbol was often different 
from any of the STAs of individual neurons (Fig. 4B). 
Thus, many distinct visual messages are represented by 
combinations of spiking and silence in the population of 
retinal ganglion cells. 

Synergy from silence depends critically on the identity 
of the two neurons: if they are functionally very similar, 
then the requirement of silence from cell B indiscrimi- 
nately attenuates all of the firing events produced by cell 
A (Fig. 4C). Now, the 1 (g) symbol encodes the same 
visual message as the single spike (Fig. 4D), but this 
compound symbol has many fewer events than either in- 
dividual spike. In fact, this combination can even result 
in a loss of information relative to that conveyed by a 
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FIG. 3 Synergy from silence. A. Distribution of synergy val- 
ues of the 1 ® symbol for natural movies (top panel, 5497 
cell pairs) , checkerboard movies (middle panel, 688 pairs) and 
uniform flicker (683 pairs) . Top panel also shows the distribu- 
tion of maximum 1 synergies for each cell found by pairing 
it with its most informative silent partner (black). B. Distri- 
bution of synergy values of 1 ® ® (top) and 1 (x) ® ® 
symbols (middle) for salamander and 1 (g) ® (8 for guinea 
pig (bottom panel) under naturalistic stimulation. C,D. Syn- 
ergy of l(g)0®0(8)0 symbols compared to the sum of synergy 
values of the corresponding three synergy of 1 (g) symbols 
for (salamander, guinea pig) under naturalistic stimulation. 
Error bars omitted for clarity. 



single spike (Fig. 4C). Information loss occurs when the 
difference between the most sharply-locked firing events 
and the typical events decreases, as can be seen in Fig. 
4C. These results demonstrate that synergy from silence 
is not a generic consequence of the decreased event rate 
for 1(8)0 symbols relative to single spikes, but instead de- 
pends on the detailed pattern of firing exhibited by the 
compound symbol. 

The lack of synergy from silence for functionally similar 
neurons was found to be systematic: the values of A/ for 
cells of the same functional subtype (see Methods) were 
found to be scattered close to zero (Fig. 4E, orange). 
Interestingly, the symbol-triggered stimulus variance for 
1 (8 symbols formed from cells of the same functional 
type was slightly but significantly greater than the stim- 
ulus variance for single spikes: variance ratio = 1.06 ± 
0.018 (S.E.M., n=71 pairs). This result is the opposite of 
what was found for synchronous spike and indicates the 
information loss here results, in part, from decreased cer- 
tainty that the stimulus is similar to the STA. We found 
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FIG. 4 Origins of synergy from silence. A. Example of in- 
formation gain from silence. Firing rate of two cells is shown 
(cell A, blue; cell B, black) along with the combination of 
cell A firing and cell B silent (bottom, pink). B. Symbol- 
triggered average for cell A (blue) and A not B (pink) against 
the other 20 cells recorded under uniform flicker (grey). C. 
Example of information loss from silence. Firing rate of two 
cells is shown (cell A, blue; cell B, black) along with the com- 
bination of cell A firing and cell B silent (bottom, orange). 
D. Symbol-triggered average for cell A (blue) and A not B 
(orange) against the other 20 cells recorded under uniform 
flicker (grey). E. Histogram of 1 8 synergies for cell pairs of 
the same function type (orange), different broad type (green), 
and similar functional type (pink). See Methods for details of 
functional classification. 



also that 1 (8) symbols formed from cells with very dif- 
ferent function, such as pairs of ON and OFF cells, were 
not synergistic (Fig. 4E, green). This is expected, be- 
cause such cells have no signal correlation and no noise 
correlation iflQi: ,32). Instead, 1 (8) symbols formed from 
similar but different neurons same broad functional type 
(e.g., fast OFF) but different subtype ('ss') usually were 
synergistic under spatially uniform stimulation (Fig. 4E, 
pink). 



Model of a Combinatorial Code 

To better understand the origin of combinatorial cod- 
ing, we constructed a simple model of retinal processing. 
We assumed that ganglion cell light responses were de- 
scribed by what is called the LN model, where the visual 
stimulus is first filtered by the classical receptive field 
and then passed through a sigmoidal threshold function 
that truncates negative values to produce a time- varying 
firing rate (see Methods) . Multiple ganglion cells had dif- 
ferent receptive fields and were assumed to be correlated 



only through their receptive field overlap (also known as 
conditional independence or lack of noise correlation). 

For this model of ganglion cell light responses, the 
classical receptive field can be thought of as a vector in 
the high dimensional space of all possible visual stimuli 
(Fig. 5A). The firing rate of each neuron depends only 
on the overlap between the stimulus and its receptive 
field, which can be described by an angle 9 between the 
stimulus and the neurons preferred direction in stimulus 
space (Fig. 5B). This overlap angle summarizes both the 
spatial and temporal integration of the classical receptive 
field. We choose the simple case of a uniform distribu- 
tion of overlap angles for the population of ganglion cells, 
allowing us to calculate the information that single cells, 
pairs and triplets convey about the visual stimulus. 

Similar to our experimental results, we found that 1(231 
symbols tend to be either redundant or nearly indepen- 
dent (Fig. 5D, left). On the other hand, 1 (g) symbols 
had many synergistic combinations and some weakly re- 
dundant ones (Fig. 5D, right). This results closely re- 
sembles the pattern of synergy from silence for different 
cell types: cells with nearly identical function (A^ ^ 0) 
were weakly redundant, ON/OFF pairs (large A9) were 
independent, and cells that were similar but different (in- 
termediate A6) almost all exhibited synergy from silence. 
Changing the neurons selectivity or noise level (see Meth- 
ods) gave qualitatively similar results (Fig. 5E-G). For 
triplets, we found that synchronized spiking 1 ® 1 (g) 1 was 
always redundant, whereas 1 8) (8 patterns demon- 
strated strong synergistic combinations and some weakly 
redundant ones (Fig. 511,1) in striking resemblance to 
the results for real ganglion cells. We emphasize that this 
model does not include stimulus dependent correlations 
between cells (also called noise correlations), showing 
that the combinatorial coding can result from stimulus- 
induced correlations alone. 



Discussion 

A combinatorial retinal code that relies on synergy 
from silence has great fiexibility. Because most ganglion 
cells respond sparsely to natural stimuli, synchronous 
spike pairs are relatively rare (Fig. 1C,D), but combina- 
tions of spiking and silence are quite common. Because 
the receptive fields of ganglion cells overlap extensively 
covering visual space ~60 times over (fs^) a single neu- 
ron can form combinatorial firing patterns with many 
nearby ganglion cells. How can the brain select interest- 
ing population symbols to recognize? Since almost all 
1 (X> (g) (g) symbols are synergistic, such symbols could 
simply be chosen at random. However, if combinations of 
spiking and silence from ganglion cells of the same func- 
tional type were avoided, the brain could significantly 
enhance the chance that remaining combinations were 
highly synergistic. Generic learning rules, such as spike- 
timing dependent plasticity, may also allow the brain to 
focus in further on the most informative retinal firing 
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FIG. 5 Synergy per symbol in a simple neural population 
model. A. Stimuli are vectors in the space of all possible 
stimuli, here depicted in two-dimensions. The preferred stim- 
ulus of a neuron is described by the direction of a unit vector 
in stimulus-space (pink arrow) , defining a spiking probability 
along this direction (grey scale). All cells in the model are 
identical except for their preferred direction. B. The spiking 
probability of the neuron from panel A shown as a function 
of the overlap between the stimulus and the cells preferred 
direction, 9. C. For two cells, the symmetry of the model im- 
plies that the joint information is only a function of the angle 
between the preferred directions of the cells, AG. D. Syn- 
ergy for 1 (8) 1 symbols (left column) and 1 (8) symbols (right 
column) shown as a function of the angle between the cells 
preferred stimuli, A6. Model parameters are /S — 8, a = 5, 
without additive background noise C = (see Methods). E. 
Same as in D, but with a higher spiking threshold (a = 8), 
which results in more selective spiking. F. Same as in D, but 
with model parameters /3 = 4, a = 2, and C = 0, correspond- 
ing to a lower selectivity and a noisier response. G. Same as 
in F, but with an additive noise ( = 0.02. H. Synergy for the 
triplet symbol 1 ® 1 ® 1 shown as a function of the angle be- 
tween cell 1 and cells 2 (x-axis), and the angle between cells 2 
and cell 3 (y-axis). Same model parameters as in D. I. Same 
model as H, but for the triplet symbol 1 ® ® . 
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FIG. 6 Schematic of circuit for recognizing compound sym- 
bols. Left: Cells A and B have excitatory synapses onto 
a readout cell with a threshold of two simultaneous inputs. 
Right: Spiking cell (A) has a direct excitatory synapse onto a 
readout neuron. Silent cell (B) excites an inhibitory cell (i), 
which feeds forward onto the readout neuron with a threshold 
of one spike. When cell B spikes, this pathway indirectly ve- 
toes excitation of the readout neuron; as a result, the readout 
neuron will only be excited when A fires but B is silent. This 
network is easily generalized to confer sensitivity to silence 
from multiple neurons by having all such neurons excite the 
same inhibitory cell (cells C, D). Such feedforward inhibitory 
circuitry has been described in the cortex (|3lh . 



patterns. 

How might the central brain recognize combinations 
of spiking and silence? We propose that they are de- 
tected by a simple etrcuit involving feedforward inhibition 
along with monosynaptic excitation (Fig. 6). Networks 
of inhibitory interneurons clearly allow cortical circuits 
to synchronize and oscillate l|35t 13^, but interneurons 
possess extreme functional diversity Is^) suggesting 
more detailed roles in information processing than cur- 
rently appreciated (39; 40). In particular, many subtypes 
exhibit very fast and reliable transmission with relatively 
little spike frequency adaptation or synaptic depression 
(fsll '41^ ■ These properties are ideal for vetoing excitation 
(l|42li ; I43|) ■ thereby detecting combinations of spiking and 
silence in the input to a cortical circuit. 

The combinatorial code of the retina combines ele- 
ments of all three hypotheses about population codes, 
but in ways that are different than previously thought 
l)l6|) . Redundancy was found in the synchronous events. 
At the same time, synchronous spikes encoded more in- 
formation than either constituent spike alone. Some of 
this additional information resulted from increased cer- 
tainty about the visual stimulus 13) ^ form of av- 
eraging over spikes to achieve higher fidelity. Ganglion 
cells of different broad functional types were found to be 
largely independent. Synergy was found in combinations 
of spiking and silence, rather than in synchronous spikes. 
Why might the retinas population code be structured in 
this complex fashion? Redundancy can benefit a neu- 
ral code by increasing the robustness of representation 
and by tagging patterns of neural activity to be learned 
l|45|) . At the same time, synergistic compound symbols 
constitute a huge vocabulary of different visual features 
multiplexed onto optic nerve fibers and represented at 
the fastest response time of the circuit. 

The simple model we presented suggests that combi- 
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natorial coding does not require elaborate circuit mecha- 
nisms or strong noise correlations, nor is it highly specific 
to the processing carried out in the retina. This implies 
that similar results may be found in other neural circuits. 
Furthermore, average synergy and redundancy have been 
reported for cortical neurons ifl^ 12(1 : . suggesting that 
these circuits can also signal with synergistic and redun- 
dant ensemble firing patterns. Because the same neu- 
rons can participate in both synergistic and redundant 
firing patterns whose contributions largely cancel, even 
the observation of average independence ifT^ [l8|) does 
not preclude combinatorial coding. Similar results have 
been found for spike patterns produced by a single neu- 
ron, where spike pairs separated by short time intervals 
were synergistic but pairs separated by longer time in- 
tervals were redundant l|27i) . As the methods we have 
used to analyze the retinal code can readily be extended 
to other neural systems, this approach promises to yield 
new insights into the nature of population neural codes 
throughout the brain. 



Methods 

Electrophysiological Recording. Eyes were dis- 
sected from the salamander or guinea pig and cut into 
pieces that left the retina and pigment epithelium in- 
tact. These pieces were placed with the ganglion cells 
facing a multi-electrode array and were perfused with 
oxygenated Ringers at room temperature (salamander) 
or Ames medium at 36°C (guinea pig). Stable recordings 
of over 12 hours were achieved under these conditions. 
Extracellular voltages were recorded by a MultiChannel 
Systems MEA 60 microelectrode array and streamed to 
disk for offline analysis. Spike waveforms were sorted us- 
ing the spike size and shape in a 2.5 ms window. Only 
well isolated spike waveforms were used: we required that 
spike trains had fewer than 0.5 % of the inter-spike in- 
terval less than 2 ms and that fewer than 1 % of the 
spikes were found within ± 0.2 ms of spikes from other 
cells. This study is based on measurements of 8210 (902) 
cell pairs between 369 (61) cells recorded from 5 (4) reti- 
nas in the salamander (guinea pig) with multiple movie 
clips. There were between 9 and 40 cells recorded per 
movie clip; some of the same cells were recorded during 
multiple clips. 

Visual Stimulation. Natural movie clips were ac- 
quired using a Canon Optura Pi video camera at 30 
frames per second. Movies were taken of woodland scenes 
and included several qualitatively different kinds of mo- 
tion: objects moving in a scene, optic flow, and simulated 
saccades. Checkerboard flicker consisted of 55 fim square 
regions on the retina that were randomly chosen to be 
either black or white every 33 ms. In spatially uniform 
flicker, light intensities were chosen randomly every 33 ms 
from a Gaussian distribution with a standard deviation 
equal to 18 % of the mean. All visual stimuli were dis- 
played on an NEC FP1370 monitor and projected onto 



the retina using standard optics. The mean light level 
was \2mW/m? at the retina. 

Functional classification. Salamander ganglion 
cells were divided into 3 broad classes using the time 
course of their spike-triggered average (STA) as well as 
their responses to diffuse steps of light. Fast OFF cells 
( 75% of population) had short latency responses to both 
the onset and offset of hght. Slow OFF cells ( 10 %) had 
longer latency responses and only responded at the offset 
of light. ON cells (~15%) only responded at the onset 
of light. These definitions correspond to those previously 
made usin g ra ndom flicker stimulation fast OFF, weak 
OFF^N In as well as using flashes ON/OFF, OFF, 
ON pB) . Fast OFF cells were resolved into 6 subtypes us- 
ing an information theoretic method of functional classifi- 
cation ifss)) ; this method is sensitive to the entire response 
function of a neuron, rather than just its spike-triggered 
average. Cells of the same subtype had nearly identi- 
cal STAs; different subtypes had characteristic shifts in 
their response latency as well as the degree to which their 
STAs were biphasic versus monophasic. 

Information carried by a neural symbol. We first 
estimated the time-dependent rate at which each neural 
symbol a occurs, rc{t) , by binning the spike train in time 
windows At and counting the number of spikes in each 
window across all stimulus trials. We then used Eqn. 1 
to estimate the information that symbol a conveys about 
the stimulus with At as a free parameter. We extrapo- 
lated to the limit At — > , giving a bin-free estimate 
of the information l)27j) . To correct for undersampling 
and bias, a jackknife approach was used, in which we 
estimated the information in subsets of the data and ex- 
trapolated to an infinite number of repeats ll|47D . We have 
omitted information values of symbols when the symbol 
rate was too low to give a reliable estimate of the in- 
formation. An alternative approach measures the infor- 
mation conveyed by individual occurrences of an event 
(^3)- Our definition instead measures the average infor- 
mation conveyed by the specific time at which a single 
event occurs. As discussed in ((27|) . these two measures 
are consistent, and Eqn. 1 can be derived in many ways, 
including as a limit of the information per event defined 
in ifisl) . We emphasize that Eqn. 1 is an exact measure 
of the information carried by the arrival time of a single 
event and makes no assumptions about the correlations 
between events. In Fig 2, we defined stimulus certainty 
by convolving the stimulus with the symbol-triggered av- 
erage, compiling the distribution of such values whenever 
the symbol occurred, and calculating the variance of that 
distribution. 

Potential for synergy. We estimated how syner- 
gistic the joint spiking symbol 1 ® 1 could have been, 
given the firing rates of its constituent cells, ri^(t) and 
riB(t) . At every time point, the 10 1 symbol rate 
could have been anywhere between the minimal value 
r™"(t) = max{Q, riA{t) + riB(t) - 1/At}, where At is 
size of the time bin, and the maximal value r^B^{t) = 
min{riA(t) , risit)}. Because information per symbol 
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is higher for larger modulations of the symbol rate, the 
maximum information will be obtained if at each time 
the rate takes either its maximal or minimal value; dif- 
ferent mean symbol rates can be obtained by distributing 
different time points between their maximal and minimal 
symbol rates. We start from r™^"(i) and add 1/NAt to 
the rate of one of the time bins (where N is number of re- 
peated presentation of the stimulus), which corresponds 
to adding one synchronous spike pair during all N stim- 
ulus trials. We choose the time bin such that this ad- 
dition results in the largest increase in information that 
would be carried by the 1 (8) 1 symbol. Then, we seek 
pairs of bins for which raising the rate in one bin and 
reducing the rate in the other bin by the same amount 
(i.e. keeping tab constant) would further increase the 
information. We stop when no such pair is found, thus 
achieving a locally optimal solution for distributing syn- 
chronous spike pairs across time to achieve the maximal 
possible information. We repeat this procedure for each 
possible value of tab (in steps of 1/NAT ) and find the 
largest 1 (g) 1 information over all the possible values of 
fAB- The potential for synergy in the 1 (8) 1 symbol is 
given by the difference between this maximum possible 
information for the 1 (8) 1 symbol and each cells individual 
information. We note that our greedy algorithm gives a 
lower bound on the synergy that the two cells firing rates 
could achieve. 

Model of ganglion cell light responses. Each gan- 
glion cell has a receptive field given by a unit vector Vi 
in the space of all visual stimuli. This preferred direc- 
tion has an overlap with all stimuli s defined by the angle 
0i = Vi- s. The spiking probability of each cell in response 
to a stimulus, P{li\s), is given by a sigmoidal function 
i [1 + tanh(/3(i/i • s) — a)] , where a is a bias term, con- 
trolling the threshold of the neuron, and /3 is the slope 
of the sigmoid, controlling how deterministic the cells re- 
sponse is (the firing probability will be either zero or one 
with a very sharp threshold). We added more noise in 
the form of a background firing rate, implemented by 
adding ( to the spiking probability (and truncating to 
the range [0, 1]). To estimate the synergy or redundancy 
of pairs and triplets of cells, we averaged responses over a 
uniformly distributed stimulus ensemble (randomly sam- 
pled with 50,000 stimuli) and a set of 100 neurons with 
preferred directions in stimulus space ranging from to 
27r in equal jumps. The joint spiking probability of cells 
A and B is equal to P{1a\s)P{1b\s). The information 
per symbol values and synergy values were calculated as 
for the retinal ganglion cells. 
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